<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  <title>IE Graph Construction &mdash; Graph4NLP v0.4.1 documentation</title><link rel="stylesheet" href="../../_static/css/theme.css" type="text/css" />
    <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  <script id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
        <script src="../../_static/language_data.js"></script>
        <script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
    <script src="../../_static/js/theme.js"></script>
    <link rel="index" title="Index" href="../../genindex.html" />
    <link rel="search" title="Search" href="../../search.html" />
    <link rel="next" title="Dynamic Graph Construction" href="dynamic_graph_construction.html" />
    <link rel="prev" title="Constituency Graph Construction" href="constituency_graph_construction.html" /> 
</head>

<body class="wy-body-for-nav"> 
  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search" >
            <a href="../../index.html" class="icon icon-home"> Graph4NLP
          </a>
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search.html" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>
        </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <p class="caption"><span class="caption-text">Get Started</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../welcome/installation.html">Install Graph4NLP</a></li>
</ul>
<p class="caption"><span class="caption-text">User Guide</span></p>
<ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../graphdata.html">Chapter 1. Graph Data</a></li>
<li class="toctree-l1"><a class="reference internal" href="../dataset.html">Chapter 2. Dataset</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../construction.html">Chapter 3. Graph Construction</a><ul class="current">
<li class="toctree-l2 current"><a class="reference internal" href="../construction.html#roadmap">Roadmap</a><ul class="current">
<li class="toctree-l3"><a class="reference internal" href="dependencygraphconstruction.html">Dependency Graph Construction</a></li>
<li class="toctree-l3"><a class="reference internal" href="constituency_graph_construction.html">Constituency Graph Construction</a></li>
<li class="toctree-l3 current"><a class="current reference internal" href="#">IE Graph Construction</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#how-to-use">How to use</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="dynamic_graph_construction.html">Dynamic Graph Construction</a></li>
<li class="toctree-l3"><a class="reference internal" href="embedding_construction.html">Embedding Construction</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../gnn.html">Chapter 4. Graph Encoder</a></li>
<li class="toctree-l1"><a class="reference internal" href="../decoding.html">Chapter 5. Decoder</a></li>
<li class="toctree-l1"><a class="reference internal" href="../classification.html">Chapter 6. Classification</a></li>
<li class="toctree-l1"><a class="reference internal" href="../evaluation.html">Chapter 7. Evaluations and Loss components</a></li>
</ul>
<p class="caption"><span class="caption-text">Module API references</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../modules/data.html">graph4nlp.data</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/datasets.html">graph4nlp.datasets</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/graph_construction.html">graph4nlp.graph_construction</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/graph_embedding.html">graph4nlp.graph_embedding</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/prediction.html">graph4nlp.prediction</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/loss.html">graph4nlp.loss</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../modules/evaluation.html">graph4nlp.evaluation</a></li>
</ul>
<p class="caption"><span class="caption-text">Tutorials</span></p>
<ul>
<li class="toctree-l1"><a class="reference internal" href="../../tutorial/text_classification.html">Text Classification Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../tutorial/semantic_parsing.html">Semantic Parsing Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../tutorial/math_word_problem.html">Math Word Problem Tutorial</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../tutorial/knowledge_graph_completion.html">Knowledge Graph Completion Tutorial</a></li>
</ul>

        </div>
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../index.html">Graph4NLP</a>
      </nav>

      <div class="wy-nav-content">
        <div class="rst-content">
          <div role="navigation" aria-label="Page navigation">
  <ul class="wy-breadcrumbs">
      <li><a href="../../index.html" class="icon icon-home"></a> &raquo;</li>
          <li><a href="../construction.html">Chapter 3. Graph Construction</a> &raquo;</li>
      <li>IE Graph Construction</li>
      <li class="wy-breadcrumbs-aside">
            <a href="../../_sources/guide/construction/iegraphconstruction.rst.txt" rel="nofollow"> View page source</a>
      </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
             
  <div class="section" id="ie-graph-construction">
<span id="id1"></span><h1>IE Graph Construction<a class="headerlink" href="#ie-graph-construction" title="Permalink to this headline">¶</a></h1>
<p>The information extraction graph (IE Graph) aims to extract the structural information
to represent the high-level information among natural sentences, e.g., text-based documents.
We divide this process into two three basic steps:</p>
<ol class="arabic simple">
<li><p>coreference resolution,</p></li>
<li><p>constructing IE relations,</p></li>
<li><p>graph construction.</p></li>
</ol>
<p>The figure below is an example for IE graph construction which contains both the Co-reference process and
the Open Information Extraction process.</p>
<img alt="../../_images/IE_graph_v1.jpg" src="../../_images/IE_graph_v1.jpg" />
<p>Coreference resolution is the basic procedure for information extraction task which
aims to find expressions that refer to the same entities in the text sequence.
For example, the noun “James” and pronouns “He” may refer to the same object (person)
in sentence “James is on shop. He buys eggs.” In this step, all pronouns in the raw paragraph
will be replaced with the corresponding nouns.</p>
<p>The code below shows the co-reference resolution process. We first use processor such as
<code class="docutils literal notranslate"><span class="pre">Standford</span> <span class="pre">CoreNLP</span></code> to obtain the co-reference dict of the raw text input.
Then, we replace the <code class="docutils literal notranslate"><span class="pre">pron</span> <span class="pre">strs</span></code> in the raw input with the <code class="docutils literal notranslate"><span class="pre">original</span> <span class="pre">entity</span> <span class="pre">str</span></code> to get the resolved text.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># Do coreference resolution on the whole &#39;raw_text_data&#39;</span>
<span class="n">coref_json</span> <span class="o">=</span> <span class="n">nlp_processor</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="n">raw_text_data</span><span class="o">.</span><span class="n">strip</span><span class="p">(),</span> <span class="n">properties</span><span class="o">=</span><span class="n">props_coref</span><span class="p">)</span>
<span class="kn">from</span> <span class="nn">.utils</span> <span class="kn">import</span> <span class="n">CORENLP_TIMEOUT_SIGNATURE</span>
<span class="k">if</span> <span class="n">CORENLP_TIMEOUT_SIGNATURE</span> <span class="ow">in</span> <span class="n">coref_json</span><span class="p">:</span>
    <span class="k">raise</span> <span class="ne">TimeoutError</span><span class="p">(</span><span class="s1">&#39;Coref-CoreNLP timed out at input: </span><span class="se">\n</span><span class="si">{}</span><span class="se">\n</span><span class="s1"> This item will be skipped. &#39;</span>
                       <span class="s1">&#39;Please check the input or change the timeout threshold.&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">raw_text_data</span><span class="p">))</span>

<span class="n">coref_dict</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">coref_json</span><span class="p">)</span>

<span class="c1"># Extract and preserve necessary parsing results from coref_dict[&#39;sentences&#39;]</span>
<span class="c1"># sent_dict[&#39;tokenWords&#39;]: list of tokens in a sentence</span>
<span class="n">sentences</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">for</span> <span class="n">sent</span> <span class="ow">in</span> <span class="n">coref_dict</span><span class="p">[</span><span class="s1">&#39;sentences&#39;</span><span class="p">]:</span>
    <span class="n">sent_dict</span> <span class="o">=</span> <span class="p">{}</span>
    <span class="n">sent_dict</span><span class="p">[</span><span class="s1">&#39;sentNum&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sent</span><span class="p">[</span><span class="s1">&#39;index&#39;</span><span class="p">]</span>  <span class="c1"># start from 0</span>
    <span class="n">sent_dict</span><span class="p">[</span><span class="s1">&#39;tokens&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="n">sent</span><span class="p">[</span><span class="s1">&#39;tokens&#39;</span><span class="p">]</span>
    <span class="n">sent_dict</span><span class="p">[</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">token</span><span class="p">[</span><span class="s1">&#39;word&#39;</span><span class="p">]</span> <span class="k">for</span> <span class="n">token</span> <span class="ow">in</span> <span class="n">sent</span><span class="p">[</span><span class="s1">&#39;tokens&#39;</span><span class="p">]]</span>
    <span class="n">sent_dict</span><span class="p">[</span><span class="s1">&#39;sentText&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">sent_dict</span><span class="p">[</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">])</span>
    <span class="n">sentences</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">sent_dict</span><span class="p">)</span>

<span class="k">for</span> <span class="n">k</span><span class="p">,</span> <span class="n">v</span> <span class="ow">in</span> <span class="n">coref_dict</span><span class="p">[</span><span class="s1">&#39;corefs&#39;</span><span class="p">]</span><span class="o">.</span><span class="n">items</span><span class="p">():</span>
    <span class="c1"># v is a list of dict, each dict contains a str</span>
    <span class="c1"># v[0] contains &#39;original entity str&#39;</span>
    <span class="c1"># v[1:] contain &#39;pron strs&#39; refers to &#39;original entity str&#39;</span>
    <span class="n">ent_text</span> <span class="o">=</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;text&#39;</span><span class="p">]</span>  <span class="c1"># &#39;original entity str&#39;</span>
    <span class="k">if</span> <span class="s1">&#39;,&#39;</span> <span class="ow">in</span> <span class="n">ent_text</span><span class="p">:</span>
        <span class="c1"># cut the &#39;original entity str&#39; if it is too long</span>
        <span class="n">ent_text</span> <span class="o">=</span> <span class="n">ent_text</span><span class="o">.</span><span class="n">split</span><span class="p">(</span><span class="s1">&#39;,&#39;</span><span class="p">)[</span><span class="mi">0</span><span class="p">]</span><span class="o">.</span><span class="n">strip</span><span class="p">()</span>
    <span class="n">ent_sentNum</span> <span class="o">=</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;sentNum&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>  <span class="c1"># the sentNum &#39;original entity str&#39; appears in</span>
    <span class="n">ent_startIndex</span> <span class="o">=</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;startIndex&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>  <span class="c1"># the startIndex &#39;original entity str&#39; appears in</span>
    <span class="n">ent_endIndex</span> <span class="o">=</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;endIndex&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>  <span class="c1"># the endIndex &#39;original entity str&#39; appears in</span>

    <span class="k">for</span> <span class="n">pron</span> <span class="ow">in</span> <span class="n">v</span><span class="p">[</span><span class="mi">1</span><span class="p">:]:</span>
        <span class="n">pron_text</span> <span class="o">=</span> <span class="n">pron</span><span class="p">[</span><span class="s1">&#39;text&#39;</span><span class="p">]</span>  <span class="c1"># &#39;pron strs&#39;</span>
        <span class="k">if</span> <span class="n">ent_text</span> <span class="o">==</span> <span class="n">pron_text</span> <span class="ow">or</span> <span class="n">v</span><span class="p">[</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;text&#39;</span><span class="p">]</span> <span class="o">==</span> <span class="n">pron_text</span><span class="p">:</span>
            <span class="k">continue</span>
        <span class="n">pron_sentNum</span> <span class="o">=</span> <span class="n">pron</span><span class="p">[</span><span class="s1">&#39;sentNum&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>  <span class="c1"># the sentNum &#39;pron str&#39; appears in</span>
        <span class="n">pron_startIndex</span> <span class="o">=</span> <span class="n">pron</span><span class="p">[</span><span class="s1">&#39;startIndex&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>
        <span class="n">pron_endIndex</span> <span class="o">=</span> <span class="n">pron</span><span class="p">[</span><span class="s1">&#39;endIndex&#39;</span><span class="p">]</span> <span class="o">-</span> <span class="mi">1</span>

        <span class="c1"># replace &#39;pron str&#39; with &#39;original entity str&#39;</span>
        <span class="n">sentences</span><span class="p">[</span><span class="n">pron_sentNum</span><span class="p">][</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">][</span><span class="n">pron_startIndex</span><span class="p">]</span> <span class="o">=</span> <span class="n">ent_text</span>
        <span class="k">for</span> <span class="n">rm_idx</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">pron_startIndex</span><span class="o">+</span><span class="mi">1</span><span class="p">,</span> <span class="n">pron_endIndex</span><span class="p">):</span>
            <span class="n">sentences</span><span class="p">[</span><span class="n">pron_sentNum</span><span class="p">][</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">][</span><span class="n">rm_idx</span><span class="p">]</span> <span class="o">=</span> <span class="s2">&quot;&quot;</span>

<span class="c1"># build resolved text</span>
<span class="k">for</span> <span class="n">sent_id</span><span class="p">,</span> <span class="n">_</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">sentences</span><span class="p">):</span>
    <span class="n">sentences</span><span class="p">[</span><span class="n">sent_id</span><span class="p">][</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="nb">filter</span><span class="p">(</span><span class="k">lambda</span> <span class="n">a</span><span class="p">:</span> <span class="n">a</span> <span class="o">!=</span> <span class="s2">&quot;&quot;</span><span class="p">,</span> <span class="n">sentences</span><span class="p">[</span><span class="n">sent_id</span><span class="p">][</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">]))</span>
    <span class="n">sentences</span><span class="p">[</span><span class="n">sent_id</span><span class="p">][</span><span class="s1">&#39;resolvedText&#39;</span><span class="p">]</span> <span class="o">=</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">sentences</span><span class="p">[</span><span class="n">sent_id</span><span class="p">][</span><span class="s1">&#39;tokenWords&#39;</span><span class="p">])</span>
</pre></div>
</div>
<p>To construct an IE graph, the second step is to extract the triples from the resolved text,
which could be completed by leveraging some well-known information extraction systems (i.e. OpenIE).
After this step, we can obtain a list of triples, and a triple can be denoted as <span class="math notranslate nohighlight">\((n_i, r_{i, j}, n_j)\)</span>.</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="c1"># use OpenIE to extract triples from resolvedText</span>
<span class="n">all_sent_triples</span> <span class="o">=</span> <span class="p">{}</span>
<span class="k">for</span> <span class="n">sent</span> <span class="ow">in</span> <span class="n">sentences</span><span class="p">:</span>
    <span class="n">resolved_sent</span> <span class="o">=</span> <span class="n">sent</span><span class="p">[</span><span class="s1">&#39;resolvedText&#39;</span><span class="p">]</span>
    <span class="n">openie_json</span> <span class="o">=</span> <span class="n">nlp_processor</span><span class="o">.</span><span class="n">annotate</span><span class="p">(</span><span class="n">resolved_sent</span><span class="o">.</span><span class="n">strip</span><span class="p">(),</span> <span class="n">properties</span><span class="o">=</span><span class="n">props_openie</span><span class="p">)</span>
    <span class="k">if</span> <span class="n">CORENLP_TIMEOUT_SIGNATURE</span> <span class="ow">in</span> <span class="n">openie_json</span><span class="p">:</span>
        <span class="k">raise</span> <span class="ne">TimeoutError</span><span class="p">(</span><span class="s1">&#39;OpenIE-CoreNLP timed out at input: </span><span class="se">\n</span><span class="si">{}</span><span class="se">\n</span><span class="s1"> This item will be skipped. &#39;</span>
                           <span class="s1">&#39;Please check the input or change the timeout threshold.&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">raw_text_data</span><span class="p">))</span>
    <span class="n">openie_dict</span> <span class="o">=</span> <span class="n">json</span><span class="o">.</span><span class="n">loads</span><span class="p">(</span><span class="n">openie_json</span><span class="p">)</span>
    <span class="k">for</span> <span class="n">triple_dict</span> <span class="ow">in</span> <span class="n">openie_dict</span><span class="p">[</span><span class="s1">&#39;sentences&#39;</span><span class="p">][</span><span class="mi">0</span><span class="p">][</span><span class="s1">&#39;openie&#39;</span><span class="p">]:</span>
        <span class="n">sbj</span> <span class="o">=</span> <span class="n">triple_dict</span><span class="p">[</span><span class="s1">&#39;subject&#39;</span><span class="p">]</span>
        <span class="n">rel</span> <span class="o">=</span> <span class="n">triple_dict</span><span class="p">[</span><span class="s1">&#39;relation&#39;</span><span class="p">]</span>
        <span class="k">if</span> <span class="n">rel</span> <span class="ow">in</span> <span class="p">[</span><span class="s1">&#39;was&#39;</span><span class="p">,</span> <span class="s1">&#39;is&#39;</span><span class="p">,</span> <span class="s1">&#39;were&#39;</span><span class="p">,</span> <span class="s1">&#39;are&#39;</span><span class="p">]:</span>
            <span class="k">continue</span>
        <span class="n">obj</span> <span class="o">=</span> <span class="n">triple_dict</span><span class="p">[</span><span class="s1">&#39;object&#39;</span><span class="p">]</span>

        <span class="c1"># If two triples have the same subject and relation,</span>
        <span class="c1"># only preserve the one has longer object</span>
        <span class="k">if</span> <span class="n">sbj</span><span class="o">+</span><span class="s1">&#39;&lt;TSEP&gt;&#39;</span><span class="o">+</span><span class="n">rel</span> <span class="ow">not</span> <span class="ow">in</span> <span class="n">all_sent_triples</span><span class="o">.</span><span class="n">keys</span><span class="p">():</span>
            <span class="n">all_sent_triples</span><span class="p">[</span><span class="n">sbj</span><span class="o">+</span><span class="s1">&#39;&lt;TSEP&gt;&#39;</span><span class="o">+</span><span class="n">rel</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">sbj</span><span class="p">,</span> <span class="n">rel</span><span class="p">,</span> <span class="n">obj</span><span class="p">]</span>
        <span class="k">else</span><span class="p">:</span>
            <span class="k">if</span> <span class="nb">len</span><span class="p">(</span><span class="n">obj</span><span class="p">)</span><span class="o">&gt;</span><span class="nb">len</span><span class="p">(</span><span class="n">all_sent_triples</span><span class="p">[</span><span class="n">sbj</span><span class="o">+</span><span class="s1">&#39;&lt;TSEP&gt;&#39;</span><span class="o">+</span><span class="n">rel</span><span class="p">][</span><span class="mi">2</span><span class="p">]):</span>
                <span class="n">all_sent_triples</span><span class="p">[</span><span class="n">sbj</span> <span class="o">+</span> <span class="s1">&#39;&lt;TSEP&gt;&#39;</span> <span class="o">+</span> <span class="n">rel</span><span class="p">]</span> <span class="o">=</span> <span class="p">[</span><span class="n">sbj</span><span class="p">,</span> <span class="n">rel</span><span class="p">,</span> <span class="n">obj</span><span class="p">]</span>

<span class="n">all_sent_triples_list</span> <span class="o">=</span> <span class="nb">list</span><span class="p">(</span><span class="n">all_sent_triples</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>  <span class="c1"># triples extracted from all sentences</span>
</pre></div>
</div>
<div class="section" id="how-to-use">
<h2>How to use<a class="headerlink" href="#how-to-use" title="Permalink to this headline">¶</a></h2>
<p>For example, we can construct the IE graph given a raw textual input:</p>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">graph4nlp.pytorch.modules.graph_construction.ie_graph_construction</span> <span class="kn">import</span> <span class="n">IEBasedGraphConstruction</span>
<span class="kn">from</span> <span class="nn">stanfordcorenlp</span> <span class="kn">import</span> <span class="n">StanfordCoreNLP</span>

<span class="n">raw_data</span> <span class="o">=</span> <span class="p">(</span><span class="s1">&#39;James is on shop. He buys eggs.&#39;</span><span class="p">)</span>

<span class="n">nlp_parser</span> <span class="o">=</span> <span class="n">StanfordCoreNLP</span><span class="p">(</span><span class="s1">&#39;http://localhost&#39;</span><span class="p">,</span> <span class="n">port</span><span class="o">=</span><span class="mi">9000</span><span class="p">,</span> <span class="n">timeout</span><span class="o">=</span><span class="mi">300000</span><span class="p">)</span>

<span class="n">props_coref</span> <span class="o">=</span> <span class="p">{</span>
                <span class="s1">&#39;annotators&#39;</span><span class="p">:</span> <span class="s1">&#39;tokenize, ssplit, pos, lemma, ner, parse, coref&#39;</span><span class="p">,</span>
                <span class="s2">&quot;tokenize.options&quot;</span><span class="p">:</span>
                    <span class="s2">&quot;splitHyphenated=true,normalizeParentheses=true,normalizeOtherBrackets=true&quot;</span><span class="p">,</span>
                <span class="s2">&quot;tokenize.whitespace&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
                <span class="s1">&#39;ssplit.isOneSentence&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
                <span class="s1">&#39;outputFormat&#39;</span><span class="p">:</span> <span class="s1">&#39;json&#39;</span>
            <span class="p">}</span>

<span class="n">props_openie</span> <span class="o">=</span> <span class="p">{</span>
    <span class="s1">&#39;annotators&#39;</span><span class="p">:</span> <span class="s1">&#39;tokenize, ssplit, pos, ner, parse, openie&#39;</span><span class="p">,</span>
    <span class="s2">&quot;tokenize.options&quot;</span><span class="p">:</span>
        <span class="s2">&quot;splitHyphenated=true,normalizeParentheses=true,normalizeOtherBrackets=true&quot;</span><span class="p">,</span>
    <span class="s2">&quot;tokenize.whitespace&quot;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
    <span class="s1">&#39;ssplit.isOneSentence&#39;</span><span class="p">:</span> <span class="kc">False</span><span class="p">,</span>
    <span class="s1">&#39;outputFormat&#39;</span><span class="p">:</span> <span class="s1">&#39;json&#39;</span><span class="p">,</span>
    <span class="s2">&quot;openie.triple.strict&quot;</span><span class="p">:</span> <span class="s2">&quot;true&quot;</span>
<span class="p">}</span>

<span class="n">processor_args</span> <span class="o">=</span> <span class="p">[</span><span class="n">props_coref</span><span class="p">,</span> <span class="n">props_openie</span><span class="p">]</span>

<span class="n">graphdata</span> <span class="o">=</span> <span class="n">IEBasedGraphConstruction</span><span class="o">.</span><span class="n">topology</span><span class="p">(</span><span class="n">raw_data</span><span class="p">,</span> <span class="n">nlp_parser</span><span class="p">,</span>
                                              <span class="n">processor_args</span><span class="o">=</span><span class="n">processor_args</span><span class="p">,</span>
                                              <span class="n">merge_strategy</span><span class="o">=</span><span class="kc">None</span><span class="p">,</span>
                                              <span class="n">edge_strategy</span><span class="o">=</span><span class="kc">None</span><span class="p">)</span>
</pre></div>
</div>
<div class="section" id="merge-strategy">
<h3>Merge strategy<a class="headerlink" href="#merge-strategy" title="Permalink to this headline">¶</a></h3>
<p>Since the ie graph is only constructed for sentences individually, we provide options to construct one graph
for the paragraph consisting of multiple sentences. Currently, we support the following options:</p>
<ol class="arabic simple">
<li><p><code class="docutils literal notranslate"><span class="pre">None</span></code>. Do not add additional nodes and edges and the original subgraphs may not be connected.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">global</span></code>. It means all subjects in extracted triples are connected by a “GLOBAL_NODE” using a “global” edge.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">user_define</span></code>. We suggest users to define their merge strategy by overriding the <code class="docutils literal notranslate"><span class="pre">_graph_connect</span></code>.</p></li>
</ol>
<div class="highlight-python notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">graph4nlp.pytorch.modules.graph_construction.ie_graph_construction</span> <span class="kn">import</span> <span class="n">IEBasedGraphConstruction</span>

<span class="k">class</span> <span class="nc">NewIEBasedGraphConstruction</span><span class="p">(</span><span class="n">IEBasedGraphConstruction</span><span class="p">):</span>
    <span class="k">def</span> <span class="nf">_graph_connect</span><span class="p">(</span><span class="bp">cls</span><span class="p">,</span> <span class="n">triple_list</span><span class="p">,</span> <span class="n">merge_strategy</span><span class="o">=</span><span class="kc">None</span><span class="p">):</span>
        <span class="o">...</span>
</pre></div>
</div>
</div>
<div class="section" id="edge-strategy">
<h3>Edge Strategy<a class="headerlink" href="#edge-strategy" title="Permalink to this headline">¶</a></h3>
<p>There are various dependency relations for ie graph nodes. Currently, we support the following options:</p>
<ol class="arabic simple">
<li><p><code class="docutils literal notranslate"><span class="pre">None</span></code>. It means we will not add additional edges.</p></li>
<li><p><code class="docutils literal notranslate"><span class="pre">as_node</span></code>. We will view each edge as a graph node and construct the bipartite graph. For example, if there is an edge whose type is <span class="math notranslate nohighlight">\(k\)</span> between node <span class="math notranslate nohighlight">\(i\)</span> and node <span class="math notranslate nohighlight">\(j\)</span>, we will insert a node <span class="math notranslate nohighlight">\(k\)</span> into the graph and link node <span class="math notranslate nohighlight">\((i, k)\)</span> and <span class="math notranslate nohighlight">\((k, j)\)</span>.</p></li>
</ol>
</div>
</div>
</div>


           </div>
          </div>
          <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
        <a href="constituency_graph_construction.html" class="btn btn-neutral float-left" title="Constituency Graph Construction" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
        <a href="dynamic_graph_construction.html" class="btn btn-neutral float-right" title="Dynamic Graph Construction" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>&#169; Copyright 2020, Graph4AI Group.</p>
  </div>

  Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
    <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
    provided by <a href="https://readthedocs.org">Read the Docs</a>.
   

</footer>
        </div>
      </div>
    </section>
  </div>
  <script>
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script> 

</body>
</html>